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Linear Algebra: Matrices* 



Louis Scharf 

This work is produced by The Connexions Project and licensed under the 
Creative Commons Attribution License * 



note: This module is part of the collection, A First Course in Electrical and Computer Engineer- 
ing. The LaTeX source files for this collection were created using an optical character recognition 
technology, and because of this process there may be more errors than usual. Please contact us if 
you discover any errors. 

The word matrix dates at least to the thirteenth century, when it was used to describe the rectangular 
copper tray, or matrix, that held individual leaden letters that were packed into the matrix to form a page 
of composed text. Each letter in the matrix, call it a^, occupied a unique position (ij) in the matrix. In 
modern day mathematical parlance, a matrix is a collection of numbers arranged in a two-dimensional array 
(a rectangle). We indicate a matrix with a boldfaced capital letter and the constituent elements with double 
subscripts for the row and column: 
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Figure 1: Matrix 



In this equation A is an mxn matrix, meaning that A has m horizontal rows and n vertical columns. 
As an extension of the previously used notation, we write A <G 1Z mxn to show that A is a matrix of size 
mxn with € 1Z. The scalar element is located in the matrix at the i th row and the j th column. For 
example, 0,23 is located in the second row and the third column as illustrated in Figure 1. 

The main diagonal of any matrix consists of the elements an. (The two subscripts are equal.) The main 
diagonal runs southeast from the top left corner of the matrix, but it does not end in the lower right corner 
unless the matrix is square (<E lZ mxm ). 
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The transpose of a matrix A £ 7^™*™ j s another matrix B whose element in row j and column i is 
bji = ciij for 1 < i < m and 1 < j < n. We write B — A T to indicate that B is the transpose of A. In 
MATLAB, transpose is denoted by A'. A more intuitive way of describing the transpose operation is to say 
that it flips the matrix about its main diagonal so that rows become columns and columns become rows. 

Exercise 1 

r 2 1 

l , then A T £ ?. Find the transpose of the matrix A - 



If Ae K ri 



Matrix Addition and Scalar Multiplication. Two matrices of the same size (in both dimensions) may 
be added or subtracted in the same way as vectors, by adding or subtracting the corresponding elements. 



The equation C = A ± B means that for each i and j, Cij 
multiplies each element of the matrix by the scalar: 



Scalar multiplication of a matrix 
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Matrix Multiplication. A vector can be considered a matrix with only one column. Thus we intentionally 
blur the distinction between lZ nxl and R n . Also a matrix can be viewed as a collection of vectors, each 
column of the matrix being a vector: 
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In the transpose operation, columns become rows and vice versa. The transpose of an n x 1 matrix, a 
column vector, is a 1 xn matrix, a row vector: 



xi 
x 2 



:x 



\X\X 2 X r , 



(3) 



Now we can define matrix-matrix multiplication in terms of inner products of vectors. Let's begin with 
matrices A £ Ji mxn an d B £ lZ nxp . To find the product AB, first divide each matrix into column vectors 
and row vectors as follows: 
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Thus a,i is the column of A and aj is the j th row of A. For matrix multiplication to be defined, the 
width of the first matrix must match the length of the second one so that all rows aj and columns bi have 
the same number of elements n. The matrix product, C = AB, is an m x p matrix defined by its elements 
as Cij — (ai,bj). In words, each element of the product matrix, cy, is the inner product of the i th row of the 
first matrix and the j th column of the second matrix. 

For n-vectors x and y, the matrix product x T y takes on a special significance. The product is, of course, 
a 1 x 1 matrix (a scalar). The special significance is that x T y is the inner product of x and y: 



[- x T -] 



Figure 3 



Thus the notation x T y is often used in place of (x,y). Recall from Demo 1 from "Linear Algebra: Other 
Norms" 1 that MATLAB uses x'*y to denote inner product. 

Another special case of matrix multiplication is the outer product. Like the inner product, it involves 
two vectors, but this time the result is a matrix: 

^'Linear Algebra: Other Norms" <http://cnx.Org/content/m21463/latest/#id2260500> 




= (x,y). 
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A = 



Figure 4 



In the outer product, the inner products that define its elements are between one-dimensional row vectors 
of x and one-dimensional column vectors of y T , meaning the element of A is XiUj. 

Exercise 2 

Find C = AB where A and B are given by 



a. A - 



1 -1 
3 



B 



-2 
4 2 
2 -2 





1 







1 


2 


3 


4 




b. A = 






, B = 



















1 




5 


6 


7 


8 






" 1 


-1 


-1 " 






" 


3 


6 " 


c. A = 


1 


-1 


1 


, B 




1 


4 


7 




1 


1 


1 






2 


5 


8 



There are several other equivalent ways to define matrix multiplication, and a careful study of the following 
discussion should improve your understanding of matrix multiplication. Consider A £ lZ mxn ,B £ lZ nxp , 
and C = AB so that C £ lZ mxp . In pictures, we have 
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(4) 



In our definition, we represent C on an entry-by-entry basis as 



Cij = (ati,bj) = Y^a ik b ky 



(5) 



In pictures, 
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Figure 5 



You will prove in Exercise 3 (Exercise ) that we can also represent C on a column basis: 

n 

c } = £ a k b k j- 

k—l 
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Figure 6 



Finally, C can be represented as a sum of matrices, each matrix being an outer product: 

n 

C= Soi/Jf 



ft 



Figure 7 



A numerical example should help clarify these three methods. 
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Let 

r 1 2 1 

2 2 1 

1 3 2 

2 1 1 

Using the first method of matrix multiplication, on an entry- by- entry basis, we have 
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or 



C = 



or 



(1 • 1 + 2-2 + 1 • 1 + 3-2) 
(2 - 1 + 1 - 2 + 2- 1 + 4-2) 
(3- 1 + 3-2 + 2- 1 + 1 -2) 



On a column basis, 



(1-2 + 2- 2+1-3 + 3-1) (1-1 + 2-1 + 1- 2 + 3-1) 
(2-2+1-2 + 2- 3 + 4-1) (2-1 + 1-1 + 2- 2 + 4.1) 
(3-2 + 3- 2 + 2- 3 + 1-1) (3-1 + 3-1 + 2- 2+1-1) 
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Collecting terms together, we have 



(1 • 1 + 2-2 + 1 • 1 + 3-2) 
C= (2-1 + 1-2 + 2-1 + 4-2) 
(3- 1 + 3-2 + 2- 1 + 1 -2) 
On a matrix-by-matrix basis, 



C = [cic 2 c 3 ] 

(1-2 + 2- 2+1-3 + 3-1) (1-1 + 2-1 + 1- 2 + 3-1) 

(2-2+1-2 + 2- 3 + 4-1) (2-1 + 1-1 + 2- 2 + 4.1) 

(3-2 + 3- 2 + 2- 3 + 1-1) (3-1 + 3-1 + 2- 2+1-1) 
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C = [123] [121] + [213] [221] + [132] + [122] [132] + [341] [111] 
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as we had in each of the other cases. Thus we see that the methods are equivalent-simply different 
ways of organizing the same computation! 

Exercise 3 

Prove that Equations 9 (5), 11 (6), and 13 (7) are equivalent definitions of matrix multiplication. 
That is, if C — AB where A £ Ji mxn and B £ lZ nxp , show that the matrix-matrix product can 
also be defined by 

n 

Cij — y^^ikbkjj (18) 
fc=l 



and, if is the k th column of C and is the k th column of A, then 



k=l 



(19) 



Show that the matrix C may also be written as the , "sum of outer products" 

n 

C = ]Ta fc /f . (20) 
fe=i 

Write out the elements in a typical outer product a/c/3j. 
Exercise 4 

Given A £ 1Z mxn , B £ 1Z pxq , and C £ 1Z TXS , for each of the following postulates, either prove that 
it is true or give a counterexample showing that it is false: 

a. (A T ) T = A. 

b. AB = BA when n = p and m = q. Is matrix multiplication commutative? 

c. A (B + C) = AB + AC when n — p^r and q — s. Is matrix multiplication distributive over 
addition? 

d. ( AB T = B T A T when n = p. 

e. (AB) C — A (BC) when n — p and q = r. Is matrix multiplication associative? 



Example 2: Rotation 

We know from the chapter on complex numbers 2 that a complex number z\ 
rotated by angle 9 in the complex plane by forming the product 



xi + jyi may be 



Z2 



(21) 



"Complex Numbers: Introduction" <http://cnx.org/content/m21412/latest/> 
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When written out, the real and imaginary parts of z-i are 

zi = (cos6 + jsinO) (x\ + jy\) 

= (cosO) x\ — (sinO) y\ + j [(sinO) x\ + (cos9) y{\ . 

If the real and imaginary parts of z\ and z-i are organized into vectors z\ and Zi as in the chapte 
on complex numbers 3 , then rotation may be carried out with the matrix- vector multiply 





X2 
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sin9 cosO 







We call the matrix R (9) 



cos9 —sin9 
sin9 cos9 



a rotation matrix. 



Exercise 5 

Let R (9) denote a 2 x 2 rotation matrix. Prove and interpret the following two properties: 



a. R T {9) =R{-6); 

b. R T (9) R{9) = R (9) R T (9) 



1 
1 



3 "Complex Numbers: Introduction" <http://cnx.org/content/m21412/latest/> 
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